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Abstract. Let T be a torsion-free uniform lattice of SU(m, 1), m > 1. Let G be either 
SU(p, 2) with p > 2, Sp(2,R) or SO(p, 2) with p > 3. The symmetric spaces associated 
to these G's are the classical bounded symmetric domains of rank 2, with the exceptions 
of SO*(8)/U(4) and SO*(10)/U(5). Using the correspondence between representations of 
fundamental groups of Kahler manifolds and Higgs bundles we study representations of the 
lattice T into G. We prove that the Toledo invariant associated to such a representation 
satisfies a Milnor-Wood type inequality and that in case of equality necessarily G = SU(p, 2) 
with p > 2m and the representation is reductive, faithful, discrete, and stabilizes a copy 
of complex hyperbolic space (of maximal possible induced holomorphic sectional curva- 
ture) holomorphically and totally geodesically embedded in the Hermitian symmetric space 
SU(p, 2)/S(U(p) x U(2)), on which it acts cocompactly. 



1. Introduction 

Let r be a (torsion free) uniform lattice in the Lie group SU(m, 1). We are interested here 
in representations, i.e. homomorphisms, of T in a Lie group G of Hermitian type, that is a 
connected semisimple Lie group with finite center and no compact factor whose associated 
symmetric space X = G/K is Hermitian (K is a maximal compact subgroup of G). We will 
always assume that G, hence X, are irreducible. The classical groups of Hermitian type are 
SU(p, q), Sp(n,M), SO*(2n) and SOo(n,2) whose associated symmetric spaces' real ranks are 
respectively min(p, q), n, [n/2], and 2. 

The Toledo invariant is a number naturally associated to such a representation p : F — ► 
G, and it has been recognized over the years to play a fundamental role. It is defined as 
follows. Let W™ = SU(m, l)/S(U(m) x U(l)) be complex hyperbolic m-space, and let / be 
any (smooth) p-equivariant map — ► X. The symmetric space X being Hermitian, it 
is a Kahler manifold, and its Kahler form u>x may be pulled-back by / to give a 2-form 
f*u)X on which goes down by p-equivariance to a form on the closed complex hyperbolic 
manifold M = r\H™. We will make no difference between T-invariant objects on and 
the corresponding objects on M. For example, f*ux is either a 2-form on or a 2-form 
on M, depending on the context. Similarly, we denote by g and u> the invariant metric and 
Kahler form of M™, as well as the induced metric and Kahler form on M. Now, the de 
Rham cohomology class in Fi 2 dR {M) defined by f*uix depends only on p, not on /, and will 
be denoted by . This class is then evaluated against the Kahler class of M to give the 

desired number 

r(p) = / (A*, oj)dV=—l' p*ux A Z" 1 
2m J M ml J M 

where p*tox is any representative of (., .) is the scalar product induced by g on 2-forms 

and dV = ^ uj m is the Riemannian volume form of (or M) . 
The Toledo invariant is of particular interest because 
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(1) it is constant on connected components of the space of representations Hom(T, G); 

(2) it satisfies a Milnor-Wood type inequality, namely r(p) is bounded in absolute value 
by a quantity depending only on the (real) rank of the symmetric space X and the volume 
of M = F\W™. More precisely, if the Riemannian metrics on W™ and X are normalized so 
that the minimum of their holomorphic sectional curvatures is —1 (so that the holomorphic 
sectional curvature of X is pinched between —1 and — 1/rkX), the following holds: 

\t( P )\ < rk^Vol(M). 

(3) maximal representations, i.e. representations p for which \r(p)\ = rkAf Vol(M), are 
expected to be of a very special kind, and therefore rigidity results should follow. 

The Toledo invariant appeared for the first time in 1979 Toledo's paper [To79] and more 
explicitly in [To89], where the Milnor-Wood inequality (2) was proved for m = 1 and rk^ = 1, 
namely T is the fundamental group of a Riemann surface and G = SU(n, 1). Toledo also proved 
that maximal representations are faithful with discrete image, and stabilize a complex line in 
the complex hyperbolic re-space X, generalizing Goldman's results for G = SL(2,R) [Go80, 
Go88]. At approximately the same time, Corlette, using a very similar invariant, the volume of 
the representation, established in [Co88] the same kind of result for rre > 2 and G = SU(n, 1). 
An immediate corollary is that a uniform lattice in SU(m, 1) can not be deformed non-trivially 
in SU(re, 1), re > rre > 1, a result first obtained by Goldman and Millson in [GM87] using 
different methods. These results have been extended to the non-uniform case in [BI05, KM04] 
(the definition of the Toledo invariant must be modified). Therefore the case where the rank 
of the symmetric space associated to G is 1 is now settled. 

Using the work of Domic- Toledo [DT87] and Clerc-0rsted [C0O1], Burger and Iozzi obtained 
in [BI05] the Milnor-Wood inequality (2) in full generality. Since then, much progress have 
been made and maximal representations of the fundamental group of a Riemann surface into 
such general groups of Hermitian type are well understood [BIW03, BIW06, BGG03, BGG05]. 

So far, the problem of representing higher dimensional complex hyperbolic lattices in Lie 
groups of Hermitian type of rank at least two seems to have been left aside. This is the 
question we (begin to) address in this paper. Since the Milnor-Wood inequality (2) is known, 
one would a priori like to focus on point (3) and give a complete description of the maximal 
representations. Our strategy will be different, and to explain it we need to say a word about 
the available methods to prove the results we mentioned. Essentially, there are two different 
ways of attacking the problem. The one used by Burger and Iozzi relies on bounded coho- 
mology theory and allows to prove the bound (2) in great generality but gives relatively few 
informations on the maximal representations, so that a separate study has to be made. The 
second one, used in [Co88, KM04] and in [BGG03, BGG05], relies on harmonic maps and/or 
Higgs bundles machinery and belongs more to the world of complex differential geometry. Fol- 
lowing this approach, as we shall, makes it quite difficult to prove the Milnor-Wood inequality 
(and in fact no such proof is known in the general case) but once it is proved (in some special 
cases), maximal representations are easier to understand. 

The Higgs bundle theory was developed by Hitchin [Hi87, Hi92] for Riemann surfaces and 
Simpson [Si88, Si92, Si94a, Si94b] in higher dimensions to study fundamental groups of Kahler 
manifolds and their linear representations. It consists in establishing a correspondence between 
representations of the fundamental group T of a Kahler manifold M and purely holomorphic 
objects, called polystable Higgs bundles over M, and then working on these objects with tools 
from complex geometry. This is in fact only possible for reductive representations, since the 
construction of Higgs bundles requires harmonic maps, but this restriction will not be a serious 
issue for our purposes. We will explain this correspondence in some details in section 2. For 
now let us simply say that to a reductive representation of T is associated a Higgs bundle on 
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M, that is a holomorphic vector bundle (E,d) on M together with a holomorphic (l,0)-form 
9, the Higgs field, which takes values in (a subbundle of) the bundle of endomorphisms of E 
and satisfies [9, 9] = 0. We shall often write 9 : E — > E®Q}, where is the sheaf (of germs) 
of holomorphic 1-forms on M, which we identify with the holomorphic cotangent bundle of M. 
Polystability is a condition that relates the slope of proper ^-invariant saturated subsheaves 
of E to the slope of E itself (which will be here). Recall that the slope of a saturated sheaf 
is defined as the quotient of its degree by its rank. 

The Toledo invariant may be defined as before for representations p of the fundamental 
group T of any closed Kahler manifold M in a Lie group of Hermitian type G ( f is then 
simply a p-equivariant map from the universal cover M of M to X = G/K), but in the Higgs 
bundles setting it is best interpreted as the degree of a complex vector bundle over M as 
follows. The Hermitian symmetric space X is Kahler-Einstein and therefore ux is up to a 
constant the first Chern class of the holomorphic tangent bundle TX of X. Therefore f*ux 
is up to a constant the first Chern class of the induced bundle f*TX over M so that r(p) 
is, again up to a constant, simply the degree of this bundle. To be more precise, if the Ricci 
curvature tensor of X is Xx gx (remember that the Riemannian metric of X is normalized 
to have minimal holomorphic sectional curvature —1), we have r(p) = — ^yf~ deg f*TX. We 
remark that if M is complex hyperbolic of complex dimension m, Vol(M) = m! ^ +1 ) degfi 1 . 
Hence the Milnor-Wood type bound (2) can be written: 



(2') 



deg/*T* 



2Xx 



<rk*^, 
m + 1 



an interpretation we will use constantly If the />equivariant map / : M — > X is chosen to be 
harmonic, the bundle f*TX — > M is constructed from holomorphic subbundles of the Higgs 
bundle E — ► M associated to the representation p. Moreover, the fact that p is not valued in 
the full general linear group but in a smaller group G of Hermitian type implies that the Higgs 
bundle E has a special structure. The idea is then that Inequality (2') will follow from this 
particular structure and the polystability condition. This implementation of the Higgs bundle 
theory has been carried out for Riemann surfaces, for example by Xia [XiOO] in some special 
cases, and more generally by Bradlow, Garcia-Prada and Gothen in [BGG03, BGG05]. The 
Kahler manifold M being a complex curve in their situation makes it quite easy to deduce (2') 
from the structure of the Higgs bundles and maximal representations can be studied in great 
details. It is for example possible to count the number of connected components of the moduli 
space of maximal representations. The reader should consult the papers [BGG03, BGG05] to 
see the strength of the method in this case. 

When M = r\H™ is a (closed) complex hyperbolic manifold of dimension m > 2, one 
expects maximal representations to be extremely restricted. In fact they should all be induced 
by special holomorphic or antiholomorphic totally geodesic embeddings of complex hyperbolic 
m-space into the Hermitian symmetric space X associated to G. 

Inequality (2) or (2') for such higher dimensional lattices turns out to be surprisingly difficult 
to prove and disappointingly we have been obliged to restrict ourselves to the case where the 
rank of the symmetric space X is 2. Our main result is 

Theorem 1.1. Let T be a (torsion-free) uniform lattice in SU(m, 1). Let G be either S\J(p, 2) 
with p > 1, SOo(p, 2) with p > 3 or Sp(2, R). Finally let p :T — > G be a representation. 

Then \r(p)\ < 2Vol(r\Hg). If m > 1 and p is maximal, namely if \r(p)\ = 2Vol(r\H^), 
then G = SU(p, 2) with p > 2m, p is reductive, faithful, discrete, and stabilizes a holomorphic 
totally geodesic copy of complex hyperbolic m-space of holomorphic sectional curvature —1/2 
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in the Hermitian symmetric space X = SU(p, 2)/S(U(p) x U(2)). Moreover V acts cocompactly 
on this copy of complex hyperbolic space. 

Although Higgs bundles are associated to reductive representations, we do not assume that 
p is reductive in the theorem. This is because every representation can be deformed to a 
reductive one, an operation that does not change the value of the Toledo invariant. Moreover 
we shall see in section 3.3.3 that non reductive representations can not be maximal. 

The theorem covers all but two of the classical Lie groups of Hermitian type G whose 
associated symmetric spaces' rank is 2. The missing ones are SO*(8) and SO*(10). In fact, 
SO* (8) and SOo(6, 2) are isogenous and have the same associated symmetric space. This means 
that if a representation V — ► SO*(8) lifts to Spin(6, 2), then projecting down to SOo(6, 2) 
gives the result for this representation as well. 

As we said, the Milnor-Wood type inequality \r(p)\ < 2Vol(M) is not new, only the proof 
is. It is given in Section 3 for G = SU(p, 2) and in Section 4 for G = SOo(p, 2). The case of 
Sp(2,R) follows from the case of SU(2,2) since Sp(2,R) C SU(2,2). 

The theorem in particular says that for m > 1 there is no maximal representations of a 
uniform lattice T of SU(m, 1) in SOo(p, 2), Sp(2,R) or SU(p, 2) with p < 2m. Our method 
indeed yields explicit better bounds on the Toledo invariant in these cases. For representations 
p : T — > SU(p, 2), the arguments of [VZ05] can be adapted to give the following, which is 
stronger than the Milnor-Wood inequality (2) exactly when p < 2m: 

Proposition 1.2. Let T be a (torsion-free) uniform lattice in SU(m, 1) and let p : T — ► 

SU(p,2), p>2, be a representation. Then \r(p)\ < Vol(T\E£). 

If p : T — ► G is a maximal representation, for which as we said G = SU(p, 2) with p > 2m, 
we will prove that there exists a maximal holomorphic or antiholomorphic totally geodesic 
p-equivariant embedding W™ — ► X = SU(p, 2)/S(U(p) x U(2)), from which the assertions 
of our main theorem follow. By a maximal embedding — ► X we mean an embedding 
whose image's induced holomorphic sectional curvature is everywhere the greatest possible, 
namely —1/2 with our normalization. See section 3.1.2 for a discussion of these embeddings 
and a description of the stabilizer in SU(p, 2) of their images in X . If / : — ► X is such 
a /3-equivariant maximal embedding, we will loosely say that p is induced by /, although / 
determines ^(7) for 7 S V only up to composition with an element of SU(p, 2) fixing pointwisely 
the image of / in X. 

The paper is organized as follows. Section 2 is an overview on how Higgs bundles are 
constructed from representations of the fundamental group of a Kahler manifold. We say a 
few words about the corresponding moduli space, the C*-action it comes with, and the systems 
of Hodge bundles that are obtained as fixed points of this action. Section 3 is devoted to the 
proof of the main theorem when the representation takes values in SU(p, 2), which is the most 
interesting case. The first subsection is expository, we give there the necessary background on 
the geometry of the associated Hermitian symmetric space. This is used in the next subsection 
to describe the particular structure of the Higgs bundles associated to such a representation. 
The third subsection contains the proof of the Milnor-Wood type inequality and the forth deals 
with maximal representations. In the fifth we prove Proposition 1.2. Finally, Section 4 follows 
the lines of Section 3 in the case of SOo(p, 2): the first subsection describes the associated 
symmetric space whereas the second is devoted to the Higgs bundles arising in this case and 
to the proof of the Milnor-Wood type inequality. 

Acknowledgments. We would like to thank Jean-Louis Clerc and Andrei Teleman for useful 
discussions and their interest in our work. 
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2. Representations of the fundamental group, flat bundles and Higgs 

BUNDLES 

In this section we give a short presentation of the links between representations of the fun- 
damental group T of a Kahler manifold M and Higgs bundles on M. To be a little more 
precise, we will explain in some details how G-Higgs bundles are constructed from reductive 
representations of T into a linear group G. There is in fact a much deeper correspondence (a 
generalized Hitchin-Kobayashi correspondence) between the moduli space of reductive repre- 
sentations and the moduli space of G-Higgs bundles over M with some stability properties. 
However, we shall not need the full strength of this correspondence (the easy direction suf- 
fices), and we will stick to what matters for our purposes. We refer to the original papers of 
Simpson [Si88, Si92, Si94a, Si94b] and to [BGG03, BGG05] for details. Our exposition owes 
a lot to [BGG]. 

Let M be a compact Kahler manifold, T its fundamental group, and M its universal cover, 
so that M = T\M. Let G be a real connected semisimple Lie group with finite center and no 
compact factor and K a maximal compact subgroup of G. Finally, let p be a representation 
r — ► G. 

2.1. Real Higgs equations. 

Let Pq be the flat principal G-bundle M x p G on M associated to the representation p. 

A metric on Pq is a reduction of the structure group G of Pq to its maximal compact 
subgroup K, namely, a -ff -principal subbundle Pk of Pq. This is the same thing as a section 
of the associated bundle Pq Xg X ~ Pq/K over M. In our setting, since Pq is flat, this 
associated bundle is isomorphic to M X p X, and a section of this bundle is given by a p- 
equivariant map / : M — ► X. In this case, the if -principal bundle G — ► X = G/K can 
be pulled-back by / to give a if-principal bundle f*G C M x G over M. This bundle goes 
down under the action of T and yields the -ff -principal bundle Pk C Pq over M. Note that 
Pq is recovered as the bundle Pk *k G associated to Pk via the action of K on G by left 
translations. 

f*G C M x G G 




P K C P g M : ► X 




M 



Let l>g be the flat connection 1-form on the trivial G-bundle MxG — ► M : if X £ TM and 
A* is the left invariant vector field on G corresponding to A G g, uq(X, A*) = A. This form 
goes down under the T-action to give the flat connection ujq on Pq. On the bundle G — > X 
we have the usual invariant connection A defined by X(A*) = A%, where A% is the ^-component 
of A G g in the Cartan decomposition g = t © p. Let luk = /*A be the induced connection 
1-form on the pull-back f*G — ► M. Again, uk is T-invariant and gives a connection 1-form 
u>k on Pk — ► M. 
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For X G T X M ', we have d x f(X) G Tf^X = T g xG/K = g±T e xG/K = g+p. Hence we can 
define a form G O 1 (M x G) ® p by 0(X, A*) = ^d/pf). If we restrict it to /*G, we have 
that Q(X,A*) = Ap. Hence, on /*G, we have lug = wjf + 0. is T-invariant and gives an 
element of Q 1 (Pk) <8> p so that log = ^K + on Pk- The form behaves well under the 
right action of K on Pk'- -R£0 = Ad(/c _1 )0. Moreover vanishes on vectors tangent to the 
fibers of Pk — ► M. Hence can be seen as a 1-form on M with values in the vector bundle 
Pk x Ad P — ► M associated to Pk via the adjoint action of K on p. One should remark that 
this vector bundle is nothing but the quotient under V of the pull-back f*TX of the tangent 
bundle TX of X. 

Let da, Fq and &k, Fk be the covariant exterior derivatives and the curvature forms 
of the connection 1-forms ujq and lvk- We have for example dc = d + ad^c) and Fq = 
dw G + \[uj g ,u g ]. 

Since log = u>k + Q is flat, we have = Fg = Fk + |[0, 0] + d^0. Decomposing according 
to g = t © p, we obtain 

/ F A - + ±[0,0]=O 
\ d A = 

Moreover, Corlette proved in [Co88] that if the representation p is reductive, that is if the 
Zariski closure of p(F) is a reductive subgroup of G, then there exists a harmonic p-equivariant 
map / : M — > X. In our setting, this means that can be chosen to satisfy the additional 
condition 

d^0 = 0. 

Summing up, we see that a reductive representation p : F — ► G is equivalent to the data 
of a connection &k on a if-principal bundle Pk — ► M and an element G il 1 (M, Pk XAdp) 
satisfying 

i^ + i[0,O] =0 

dx© = (real Higgs equations) 
d^0 = 

So far we have not taken into account the fact that M is a Kahler manifold. 

2.2. Complex Higgs equations. 

Let Gc, Kc and 0c = tc © Pc be the complexifications of G, K, and 0. 

The if-principal bundle Pk can be extended to a i^c-principal bundle Pk c = Pk *k Kc- 
We still denote by &K the extension of the covariant exterior derivative to Pk c - can be 
extended by C-linearity to an element of U^(M, Pk c XAd Pc) = ^°(T<£M ® (Pk c ><Ad Pc))- 
Decomposing according to types we write d A = d^- + d^- and = 1,0 + ' 1 . We set 
= 1 ' g 1: °(M, P Kc XAdPc)- If we call r the involution of defined by r{A) = - f A, then 
= — t(0), for comes from a real form. 

It has been shown by Sampson [Sam86] that the harmonicity of the map / and the fact that 
X has non positive complexified sectional curvature implies that / is pluriharmonic, which 
can be written d^- 1 ^ = 0. Moreover the complexified sectional curvature of X vanishes on the 
image of the (l,0)-part of /, i.e. on the image of 0. Since here X is a symmetric space, the 
curvature is given by the Lie bracket and the vanishing of the complexified sectional curvature 

just means that [0,6] = 0. From this it follows that (d^- 1 ) = 0, namely that d^- 1 defines a 
holomorphic structure on the principal bundle Pk c — ► M and on the associated complex 
vector bundle Pk c x Ad Pc ■ The pluriharmonicity of / therefore means that 6 is holomorphic 
for this holomorphic structure. 

Looking at the real Higgs equations, we also obtain that F^ 1 — [6, t(9)] = 0. 
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Therefore, starting from a reductive representation p, we have constructed a Xc-P rm cipal 
bundle Pk c = Pr x k Kc over M equipped with a complex structure 3 = d^- 1 , and a section 
9 of (8) (Pkc x Ad Pc), where fi 1 denotes the holomorphic 1-forms on M, satisfying 

f [6>, 6>] = , tt • ■ n 

| p 1 ' 1 - {8 t{9)} = ( - complex Hlggs ec l uatlons ) 

2.3. Higgs bundles and stability conditions. 

We will call the data of a -fTc-P rm cipal bundle Pk c together with a holomorphic structure 
3 and a section 9 of fi 1 ® (-Pkt c XAdpc) satisfying [9,9] = a G-Higgs principal bundle on M. 
This is a purely holomorphic object. The section 9 is called a Higgs field. 

The remaining differential geometric data, namely the reduction Pk of the structure group 
of Pk c to K and the connection d# on Pk such that 3 = d^- 1 and Frf — [9, t{9)} = 0, can be 
rephrased in holomorphic terms with the help of adapted notions of stability for associated 
vector bundles. If E is a complex vector space on which Gc acts, we can form the associated 
vector bundle E = Pk c ^k c E. The holomorphic structure on Ppc c induces a holomorphic 
structure 3 on E. Moreover, pc can be seen as a subspace of End(E), hence we may consider 
9 as a holomorphic (l,0)-form on M with values in a subbundle P of the bundle End(E) (we 
will write 9 : E — > E ® Q 1 ). We will call the holomorphic bundle {E,3) together with 9 a 
G-Higgs vector bundle. We will often abbreviate (E,3,9) by (E,9). 

Now the connection dx induces a connection on E compatible with its complex structure 
and the curvature of this connection can be used to compute the degree (and the slope) of 
saturated subsheaves of E. The condition Fg 1 — [9, t(9)] = implies that E is polystable in 
the following sense [Si88] . 

A G-Higgs vector bundle (E, 3, 9) is called stable (resp. semistable) if for every saturated 
subsheaf T of E such that 9(F) C T ® n l and < rkJF < rk£, the slope n(T) = of T 

is smaller (resp. not bigger) than the slope \i{E) = -^M °f E. It is called polystable if it is 
the sum of stable G-Higgs vector bundles of the same slope. 

A subsheaf T of E such that 9(T) C T ® fi 1 is called a ^-invariant subsheaf or a Higgs 
subsheaf of E. Note that if the Higgs bundle E is polystable and T is a proper Higgs subsheaf 
of E with [i[T) = fJ>(E) then T is in fact a Higgs subbundle of E and E splits as the direct 
sum of T with another Higgs subbundle of the same slope [Si88]. 

2.4. Moduli space, C*-action and systems of Hodge bundles. 

Let Pk c be a fixed i^c-P rm cipal bundle on M, E a fixed vector space over C on which Gc, 
hence Kc, acts, and let E = Pk c x^ c E be the associated vector bundle. Consider the space 
of all holomorphic structures 3 on Pk c XAd/f c 6c and all (l,0)-forms 9 on M taking values in 
the bundle P = Pk c ><AdK c Pc C EndE such that: 

- 9 is holomorphic w.r.t. the complex structure induced on P by <9; 

- the complex structure on E induced by 3, still denoted by <9, turns (E, 3, 9) into a 
polystable G-Higgs bundle. 

The group J#c = Pk c x Ad-Kc of gauge transformation of Pk c acts on this space of polystable 
G-Higgs vector bundles (E, 3, 9) by pull-back of the holomorphic structure and conjugacy of 
the Higgs field. There is a corresponding moduli space M. (at least if M is a projective variety, 
see [Si92, Si94b]), which has the structure of an analytic space. 

A very important feature of this moduli space is that it comes with a natural C*-action 
given by t.[E, 3,9] = [E, 3,t9] for t £ C*. Moreover, Simpson proved in [Si94b] that for any 
[E, 3, 9] E M., the limit of [E, 3,t9] as t G C* goes to zero exists and is unique. The limit is 
therefore a fixed point of the C*-action, and this implies that it has the structure of a so-called 
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system of Hodge bundles [Si88, Si92]. More precisely, this means that E with the limiting 
holomorphic structure splits holomorphically as a sum E\ © . . . © E^ of holomorphic vector 
bundles and that the limiting Higgs field in P is given by a collection of holomorphic maps 
Oi : Ei — > Ei + i ® Q 1 , 1 < i < k (with the convention that E^+i = {0}). We will abuse 
notation and use the following kind of diagram: 

El -^E 2 -%... 6 ±lE k -^0 
to denote such a system of Hodge bundles. 

3. The case G = SU(p,2) 

In this section we prove the main theorem for representations into G = SU(p, 2). However, 
some arguments and results are valid in the general case G = SU(p, q), p > q, and therefore 
we will specialize to the case q = 2 only when necessary. 

3.1. The Hermitian symmetric space SU(p, <?)/S(U(p) x U(g)). 

3.1.1. General facts. 

The reader should consult [HeOl, Sat80] for details about this section. 

Let E be a complex vector space of dimension p + q, with p > q > 1, endowed with a 
non-degenerate Hermitian form F of signature (p, q). Let W be a g-dimensional complex 
subspace of E on which F is negative-definite, and let V be its F-orthogonal complement, so 
that E = V © W. The symmetric space X is defined as the space of all g- dimensional complex 
subspaces of E on which F is negative-definite. It is an open submanifold of the complex 
Grassmannian of g-planes of E. When q = 1, X is complex hyperbolic space of (complex) 
dimension p which we denote by H^. 

After an appropriate choice of basis, we see that the group G = SU(p, q) acts transitively 
on X by analytic isomorphisms, while the isotropy subgroup K of G at W is identified with 
the maximal compact subgroup S(U(p) x \J(q)) of SU(p, q), so that X can be identified with 
SU(p,g)/S(U(p) x%)). 

Let g be the Lie algebra of G, 6 C g the Lie algebra of K and g = t © p the corresponding 
Cartan decomposition. We have the following matrix expressions: 

t° A o ) > A G M pM} -m Hom c (W,V). 

The tangent space T Q X at o £ X will be identified with p. More generally, the tangent 
bundle TX of X is the bundle Gx^kP associated to the K -principal bundle G — > X = G/K 
via the adjoint action of K on p. The complex structure J on T Q X is given by 

/ A \ _ f _ iA 
J \ l A J ~ \ -i l A 

whereas the G-invariant Kahler metric gx on X is defined at o by 

g x (X,Y) = 2tr(FX) = 4Retr { l BA) , if X = ( ^ q ) ' F= ( 4 o) Gp ' 
The corresponding Kahler form will be denoted by ojx = 9x(J-, •)• 
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The complexifications Gc of G and Kc of K are respectively SL(p + q, C) and S(GL(p, C) x 
GL(q, C)). The Lie algebra gc of Gc splits as tc © pc where tc is the Lie algebra of lie and 

Pc = P©C=|(^° q ^ ,Ae M P| ,(C) , B G M ?)P (C)| ~ c Hom c (W, V)©Hom c (V, W). 

The extended complex structure J © Id acting on pc has two eigenspaces 

A 



and 



,0,1 






A 



AeM m (C)\ ~ c Homc(W,V) 



, AGM q}P (C)\ ~ c Hom c (V,W). 



The complexified tangent bundle T C X of X is isomorphic to G ><AdK Pc — (GXk Kc) x AdK c 
pc, whereas the holomorphic tangent bundle T 1,0 X is isomorphic to G ><AdK p 1 ' - (Gxj<- 
Kc) x AdK c P 1,0 - There is a natural Hermitian metric on the holomorphic tangent bundle of 
X given on To'°X ~ M m {€) by h(A, B) = 4tr (?BA). 

The holomorphic sectional curvature for the complex line (X) generated by a nonzero 



X 



A 



A 



l A I ^ T X, or equivalently by a non-zero I ^ ^ ] G Tq 1 ' ^, is given by 



tr((*AA) ; 



v2 ' 



(tr 

This formula shows that K((X)) is pinched between —1 and —l/q and that K((X)) = —1/q 
if and only if the column vectors of A are pairwise orthogonal and have the same norm (for 
the standard Hermitian scalar product in C p ). 

The metric gx is Einstein and with our normalization, its Ricci curvature tensor is 



P+Q 



3.1.2. Maximal embeddings. 

There is a natural identification of X 



SU(p,g)/S(U(p) x U(g)) with the space {Z € 



Mp !q (C) , I q — l ZZ > 0} [Sat80]. Therefore if m < p/q, we have a holomorphic totally 
geodesic embedding of complex hyperbolic space = SU(m, l)/S(U(m) x U(l)) into X 
given by 



B z 



( zi \ 

Z2 



( 



Z 



Z2lq 



Zralq 



\ 



€ X. 



This shows that X contains totally geodesic copies of complex hyperbolic m-space of holo- 
morphic sectional curvature —l/q, for all m < p/q. The next lemma implies that [p/q] is the 
maximal possible dimension of such submanifolds: 

Lemma 3.1. Let S be a complex linear subspace ofTo'°X. If, for every nonzero A G S, 
K((A)) = -l/q, then dim c 5 < p/q. 

Proof. The metric and the holomorphic sectional curvature are obviously invariant under the 
left action of U(p) on To' X ~ Af p , g (C). 

Let {Ai, . . . , Ad} be an orthonormal basis of S. We are going to show that there exists 
U G U(p) such that 
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/ 0( fc _i) M 



UA k = — | Ig , k = l,...,d. 

Op— fcg,g 

Since the column vectors of each Ai are pairwise orthogonal and have the same norm, there 



2^ 

3 of ea 

exists U\ G U(p) such that 

XJ X A 



2^9 



Now, for any (A, p) G C J \{(0, 0)}, * (A^4i + pA 2 )(\Ai + /xA 2 ) must be a (nonzero) multiple of 
7 g . This implies that for any (A,/i), Xp t AiA 2 + Ap^4i VI2 is a multiple of I g (because t A\A\ 
and t A 2 A 2 are). Moreover, it is trace free because A\ and ^2 are orthogonal. Thus, t A\A 2 = 
(that is each column vector of A\ is orthogonal to every column vector of A 2 ) and there exists 
U 2 G U(p) such that C/ 2 ^i-4i = UiA x and 



Wi^2 = — J 



2^ 







One might continue this process and after d steps, one obtains U = Ud . . . TJ\. It is then 
clear that d must be less than or equal to p/q. □ 

The embedding (★) will be denoted by f max and called the maximal embedding of W™ 
into X. This is because f m ax*gx = Q9, that is, for any x G M m and any X G T X W™, 
gx(dfmax(X),df max (X)) = qg(X,X), while for a general holomorphic map / : H™ — > X 
we only know that f*gx < qg from the Ahlfors-Schwarz-Pick lemma (see for example [R08O, 
Theorem 2]). Moreover, 

Proposition 3.2. Let f : — ► X = SU(p, g)/S(U(p) x U(g)) 6e a holomorphic map such 
that f*gx = Qg holds everywhere. Then p/q > m and f is totally geodesic. In fact, up to 
composition with an isometry of X , f is the maximal embedding fmax- 

Proof. If X is a nonzero tangent vector at x G M m , we denote by C X: x the complex geodesic 
through x that is tangent to X. Let z be a (global) complex coordinate on C X) x and let 
p 2 \dz\ 2 (resp. a 2 \dz\ 2 ) be the Hermitian metric induced by g (resp. f*gx = Q g) on C X: x- The 
Gaussian curvature of p (resp. a) is given by K = — ^Alogp (resp. k = — ^ A logo"). Since 
C Xt x C is totally geodesic, we have K = — 1. Moreover, because of the holomorphicity 
of /, & is bounded from above by —l/q which is the maximum of the holomorphic sectional 
curvature on X and k = — l/q iff the restriction of / to C X) x is totally geodesic. But a = ^fqp 
and so 

k = k Alogcr = x Alogp = — „ p 2 K = --. 

a z qp z qp A q 

Thus, k = —l/q and, since this is true for any (x,X), f must be totally geodesic. 

Let o = mi i (resp. d = Pt q) be fixed origins in W™ (resp. in X). One may suppose 
(after composition with an isometry of X) that f(o) = d . A consequence of the preceding 
discussion is that df(T W™ ) is a m-dimensional complex subspace of T /X on which the 
restriction of the holomorphic sectional curvature is constant, equal to —l/q. By the proof of 
Lemma 3.1, we know that m > p/q and that, after composition of / with a suitable isometry, 
d/|T D H™ = dfmax\T w^ ■ By uniqueness of the totally geodesic map satisfying this condition, 
one has / = f max . □ 
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by 



Another maximal embedding of HK 1 into X 



SU(p, q)/S(V(p) x \J(q)) (p/q > m) is given 



f" 

J m 



Zl 
~2 



z 



y z m J 



z 









z 






\ 



\ 





z 


o J 



From the previous proposition (it can also be easily verified by hand), this embedding is equal 
to fmax composed with an isometry of X . The geometric picture is maybe clearer here: f max 
is a diagonal embedding of H™ into (H™) 9 C X corresponding to a (diagonal) embedding 
of SU(m, 1) into SU(m, l) q C SXJ(p,q). Moreover the stabilizer in SU(p, q) of the image of 
f max can be computed quite easily. First, let us consider the subgroup of U(p, q) consisting of 
elements of the form: 



/ A n ■ 


■ A lg 





fill • 


■ Big 


\ 

















■ A qq 





Bql ■ 


■ Bqq 




• 





u 


• 







C n ■ 







du ■ 


• dl q 

















V c ql ■ 


• C qq 





dql ■ 


dqq 


J 



where A {j G M m (C), e M mA (C), Cy G Mi )ro (C), d 



G C and [/ S U(r) 



p — qm). 



Let us denote by S q the symmetric group on q letters, and by U(l) <? x S q the semi-direct 
product of U(l) 9 by 5^ given by the group operation (a, <r).(/3, r) = (a.a(f3),r o a). 

Define a group homomorphism <p of (U(l) 13 x 5 g ) x SU(m, 1) x U(r) in the above subgroup 
of U(p, g) in the following way: if a = (a\, . . . , a g ) G U(l) 9 , cr G 5 ? , u £ U(r) and 



.9 



.4 
C 



D 
d 



G SU(m, 1) (where A G M m (C), fi G M m ,i(C), C7 G Mi,. 



ouA, B 



(C),deC), 

OiB, 



icr(i) 



the image of (a,a,g,u) in ~U(p,q) is the matrix defined by A ic r i \ 
Cicr(j) = ^io-(i) = U = u, and the other coefficients are zero. 

Then Kerip is isomorphic to Z/ (m + 1)Z and the stabilizer in SU(p, q) of the image of f' n 
is luxip Pi SU(p, <?). 



3.2. Toledo invariant and SU(p, 2)-Higgs bundles. 

Now we consider a reductive representation p of a torsion- free uniform lattice of SU(m, 1), 
m > 1, into the Lie group of Hermitian type G = SU(p, q), p > q > 1. Let M be the closed 
complex hyperbolic manifold r\H^. As explained in the introduction, the Toledo invariant 
can be expressed using the degree of the pull-back of the holomorphic tangent bundle T 1,0 ^ 
of X = G/K by any p-equivariant map / : H™ — ► X, which we can choose to be harmonic. 
Let then (Pk c ,6) be the G-principal Higgs bundle on M associated to p and / as in section 2 
and let E be the holomorphic vector bundle on M associated to Pk c via the action of 
on E = V © W. Since Kc respects the decomposition E = V © W, the bundle E splits 
holomorphically as the sum of the rank p subbundle V = Pr c x k c ^ with the rank q subbundle 
W = Pr c Xk c W. As a differentiable bundle E is the bundle associated to the flat principal 
bundle Pq via the action of G on E: it is flat, hence of degree 0. In particular, deg V = — deg W. 
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The Higgs field 6 is a holomorphic (l,0)-form taking values in the bundle Pk c X-AdK c Pc 
Hom(W, V) © Hom(V, W) so that we can write (see also [XiOO, BGG03]) 

(3 \ , f $ : W — >V®n x 
where ' 



Recall that the Higgs vector bundle (E, 0) is polystable. 

It is clear that the bundle f*T 1,0 X is nothing but the bundle Hom(TV, V) and therefore 
its degree is simply given by pdegW* + qdegV = — (p + q)degW. We obtain that r(p) = 
^degW, so that the Milnor- Wood type inequality that should hold reads 

IdegWl < — — degQ 1 . 
m + 1 

Therefore, for q = 2 and p reductive, our main theorem can be reformulated: 

Theorem 3.3. Let V be a torsion free uniform lattice in SU(m, 1), m > 1, and let p : T — > 
SU(p, 2) be a reductive representation. Let E = V© W be the SU(p, 2)-Higgs vector bundle on 
M = T\W™ associated to p. Then |deg W\ < ^-j- deg J7 1 with equality if and only if m < p/2 
and, up to conjugacy, p is induced by the maximal embedding f ma x '■ HJ? — ► X or by its 
conjugate. 

If we deform the Higgs bundle (E,0) via the C*-action on the moduli space as in section 2, 
we obtain a system of Hodge bundles: 

El ^E 2 ^... 9 -^E k ^0. 

Moreover, each subbundle Ei splits as E{ = V{ © Wi with V{ C V and Wi C W, and 9i 
decomposes as 7, © Pi, where 7« : Vi — > Wi + \ © J7 1 and : Wi — ► Vi+i © O 1 . We obtain 
two Higgs subbundles 

Vi — > w 2 — > v 3 — > W A — > ... — > 

and 

Wi — > v 2 — >w 3 — >V A — ► . . . — > 

which are again polystable of degree 0. 

So far everything we said was valid in the general rank q case. Now we will need the 
assumption that q = 2 to ensure that the systems of Hodge bundles we obtain are simple ones 
and/or that the decomposition of W is maximal in the sense that W splits into a sum of line 
bundles. Indeed, if W has rank 2, we see that by regrouping and renaming the subbundles if 
necessary, we can write our system of Hodge bundles either as a polystable Higgs bundle of 
the form 

Vi W V 2 
with V% © V 2 = V, or as a polystable Higgs bundle of the form 

Vl Wl V 2 ^ W 2 V 3 ^ 
where W\ and W 2 are line bundles, W\ © W 2 = W and V\ © V 2 © V3 = V. 

3.3. Proof of the Milnor- Wood type inequality. 

The case of non reductive representations will be postponed to Paragraph 3.3.3. Until then, 
the representation p : G — ► SU(p, 2) is assumed to be reductive so that we can apply the 
results of Section 3.2. Our proof of the inequality |degTy| < ^j-j-degO 1 will be different 
according to the form of the system of Hodge bundles we obtain by deforming the polystable 
Higgs bundles E = V® W via the C*-action. Note that the deformation changes the holomor- 
phic structures of E, V and W, but not their isomorphism classes as differentiable complex 
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vector bundles, hence their degrees remain unchanged. During the proof, we will see that if 
equality holds, some bundles have stability properties (in the usual sense) that will be useful 
for the study of maximal representations in Section 3.4. 

We refer to [VZ05, HL97] for general facts about sheaves and stability. 

3.3.1. System of Hodge bundles of the type V\ — >W — >Vi — >0. 

Here the important point is that the system of Hodge bundles we are dealing with is a 
ternary bundle, and no limitation on the rank of W is needed. Hence the results of this 
paragraph are valid in the general case ikW = q > 1. 

Let T be the maximal destabilizing subsheaf of W, that is, the first term in the Harder- 
Narasimhan filtration of W [VZ05, HL97]. By definition, T has maximal slope among the 
subsheaves of W, hence is semistable. Consider the restriction [3jr : JT © T 1 — > V2. Since 6 
vanishes on V2, T © is a Higgs subsheaf and hence by stability, deglm/3f < —degT. 

Now, the tensor product of two semistable sheaves is again semistable and hence T © T 1 
is semistable. Therefore we have ^(J 7 ) + n{T l ) = © T 1 ) < fi(lmf3^) which implies 
{rkfc + rkT)n{F) < rk/^ft 1 ). Thus, 

degW < qui?) < g— — — — < — rdegft 

rk ftp + rk J- m m + 1 

where the first inequality follows from the fact that T is of maximal slope among the subsheaves 
in W, and the last from rk/3jp < mrk.F. 

The remaining inequality is obtained exactly in the same way by considering the dual Higgs 
bundle 

V* -^w*^+ Vf— »0. 

Assume that equality holds, for example that degW = ■^^■degQ, 1 . Then, retracing our 
steps, we see that W must be a semistable bundle (in the usual sense), that we must have 
rk (3 = mq, i.e. (3 : W <g> T 1 — > V generically injective, and moreover that deg W © Im (3 = 0. 
This last fact implies by polystability that E splits as the sum of (W©Im/3, (3) with an other 
polystable Higgs bundle E' of degree 0. In our case, this means that V2 splits holomorphically 
as lm(3 © V 2 ' and that 71 vanishes. E' is then the polystable (in the usual sense) subbundle 
V x © V{ of V. 

In the same manner, we find that deg V = ^^-deg implies that W is a semistable bundle, 
(3 = 0, V\ splits holomorphically as Ker7i © V[, and 71 : V{ — > W © O 1 is an isomorphism. 
Our system of Hodge bundle is the sum of the Higgs bundle (V[ © W, 71) with the polystable 
subbundle Ker 71 ffi V% of V. 

3.3.2. System of Hodge bundles of the type V\ — >W\ — >Vi — >W2 — > V3 — > 0. 

Here, we need to assume that rkW = q = 2, namely that W\ and W2 are line bundles. 

Assume first that f}\ : W\ © T 1 — > V2 vanishes. We then have to deal with the sum of two 
polystable Higgs bundles of degree 0: V\ — ► W\ — > and V2 — > W2 — ► V3 — > 0. In this 
situation, we already know from 3.3.1 that < degVi < ^-^deg^l 1 and |deg(V2 © ^3)! < 
^pj-degQ 1 which gives the result. 

We are left with the case where f3\ : W\ © T 1 — > V2 is non zero. 

Consider 71 : V\ — > W\ © ft 1 . We have deg Vi = deg Ker 71 + deg Im 71 < rk7i ji(Wi ©SI 1 ) 
since Ker 71 is ^-invariant and W\ © fi 1 is a semistable bundle, being the product of a stable 
bundle by a line bundle. Since deg W\ = —(deg V\ + deg V2 + deg W2 + deg V3) we obtain 

deg^ < degnl - -^^(degV 2 + degW 2 + degV 3 ). 

1 + rk 71 m 1 + rk 71 
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In the same way, 

degy 2 < - rkl " dGgQl - - ^ (degVi + degWi + degy 3 ). 
1 + rk 72 m 1 + rk 72 

Hence, 

rk7i rk72 \ degfJ 1 

deg V < — — + — — + 

1 + rk 71 1 + rk 72 / m 

^ (deg V 2 + deg W 2 + deg V 3 ) - rk ? 2 ) deg V 3 . 



1 + rk 72 1 + rk 71 / \ 1 + rk 72 

Now the commutation relation [0,9] = gives us control on the rank of the 7j's: 

Lemma 3.4. Assume that ft : Wj <8> T 1 — > Vi + \ is non zero. Then the rank of 7.; : V{ — ► 
Wi (g> is at most 1. 

Proof . This is linear algebra. We work in a single fiber above some point in M . We write 
Wi = Cwi. Since ft is non zero, there exists Z G T 1 such that (3i(Z)wi / 0. Assume that 
the rank of ji is at least 2 at some point. Then we can find two linearly independent forms 
a, a' and two vectors v,v' in Vi such that ji(v) = Wi ® a and 7i(V) = Wi <8> a'. Now, the 
commutation relation [9, 9] = means in particular that for all X,Y G T 1 and all it G V , 

&(x)7i(y)u = #p07iCx>. 

If we take u = v, Y = Z and X such that a(A) 7^ 0, we get a(Z)(3i(X)wi = a(X)f3i(Z)wi, 
which implies that a(Z) and ft(X)u;j are different from zero. So if now X' is such that 
a(X') = 0, Pi(X')wi = 0. We can choose such an X' with the additional property that 
a'(X') / 0, since a and a' are independent. This is a contradiction since we could have taken 
u = v' to prove that fc{X')wi ^ if a' (A') / 0. □ 

We assumed that ft : Wi <g> T 1 — ► V 2 is non zero, hence rk7i < 1. If rk72 > rk7i, using 
the fact that deg V 2 + deg W 2 + deg V3 and deg V3 are both non positive, and since rk72 < m, 
we get 

lm[m +1) m + 1 

and the last inequality is strict as soon as m > 1. If ik^ 2 < rk7i, that is if rk7i = 1 and 
rk72 = 0, we are in the case V\ — > Wi — ► V 2 — ► and hence 

deg V < degiVi © V 2 ) = -degWi < — -deg ft 1 . 

m + 1 

Once again, the remaining inequalities are obtained by looking at the dual Higgs bundle 

v * w * X v * w* X i?— o. 



Assume we are in the equality case and m > 1. If deg F = ^qrj-deg fi 1 , ft : Wi (8> T 1 — > V 2 
vanishes, and the equalities degVi = ^q^-degO 1 = deg (V 2 © V3) hold. We saw in 3.3.1 
that in this situation, ft = and there exists holomorphic subbundles V( C Vi such that 
7i : V- — ► Wi © are isomorphisms for i = 1, 2. 

If degW = ^q-j-degfi 1 , we find that for i = 1, 2, 7, = and there exists a holomorphic 
subbundle V? C Vj such that ft : Wi ® T 1 — ► V^' is an isomorphism. 

In either cases, deg Wi = degW2 and W with the deformed complex structure is polystable 
hence semistable. 
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3.3.3. Non reductive representations. 

Assume now that the representation p : T — > G = SU(p, 2) is not reductive. This implies 
that p(F) fixes a point £ on the boundary at infinity X(oo) of X ([La91]). Let us fix an origin 
o E X and let c be the unit speed geodesic ray starting from o representing £. Let q = t © p 
be the Cartan decomposition of g associated to o and let X £ p be such that X = c(0) in the 
usual identification of p with T Q X. We have the following description of the stabilizer of £ 
in G (see for example [Eb96]): 

Gt = {g £ G\ lim exp(-tX) g exp(tX) exists} = ATe.Ac.iVe 

t — S- + O0 

where At = exp({Y £p\[X,Y] = 0}), Ne = {g £ Ge | lim exp(-tX) g exp(tX) = 1}, and 

t — » + 00 

K't = Gz n K. 

By assumption p(T) C and we can consider the so-called semi-simplification p ss of p 
which is defined by p ss {l) = lim<->+oo exp(— tX) pi^f) exp(iX) £ K^.A^ for all 7 £ Y. The 
representation p ss belongs to the connected component of p in the space Hom(T, G) and is 
reductive: we can apply the results of the last paragraphs to get the Milnor- Wood bound on 

T {p) = T(p ss ). 

In fact we can do better. The representation p ss stabilizes the orbit K^.A^.o = A^.o, which 
is a totally geodesic submanifold of X. It is not difficult to see (and probably well known) 
that this orbit is either a totally real totally geodesic submanifold of X (for example, if c is 
a regular geodesic, it is the unique maximal flat, isometric to M 2 in our case, containing c), 
or the Riemannian product of R with a totally geodesic copy of complex hyperbolic space 
H^. _1 (of induced holomorphic sectional curvature —1). In the first case the Toledo invariant 
is zero since the restriction of the Kahler form ujx to a totally real submanifold vanishes. In 
the second one it is bounded (in absolute value) by Vol(M). 

Therefore non reductive representations can not be maximal. 

3.4. Maximal representations. 

Thanks to the previous paragraph, we know that if the representation p is maximal, it is 
reductive. Therefore we may consider the polystable Higgs bundles (E = V(B W, 9) associated 
to p. 

In order to prove the Milnor- Wood type inequality |deg W\ < deg Q 1 , we have deformed 
the Higgs bundle E to a system of Hodge bundles. Here, we need to distinguish between these 
two Higgs bundles, and we will call the latter (Eq = Vq © Wq,9o). Let d\y and dw be the 
complex structure of W and Wo- Again, although the complex structure is (by definition) not 
modified by the C*-action, in the limit dw is a priori different from dw- In fact, all we know 
is that dw is in the closure of the orbit of dw under the group of gauge transformations: i.e. 
there exist gauge transformations g^ such that g^dw goes to dw when ti goes to 0. Let us 
call Wtf the bundle W with the complex structure g*.dw- 

The main point of Section 3.3, apart from the proof of the inequality itself, was that 
|degW| = ^q-j-degO 1 implies that the bundle Wo is semistable (in the usual sense). This 
implies that W itself is semistable (regardless of the rank of W): 

Lemma 3.5. Assume that W with its initial complex structure dw is not a semistable bundle. 
Then Wq, that is W with the complex structure dw , is not semistable either. 

Proof. Let T be a subsheaf of (W, dw) such that p{T) > p(W). Let r be the rank of T. 
We have a monomorphism of sheaves T — ► W and therefore, for all i £ N, we obtain a 
monomorphism of sheaves T — ► W^. This gives non trivial holomorphic maps between the 
determinant bundle detJT = (/\ r F)** of T and /\ r W ti . This means that for all i £ N, the 
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cohomology group H°(M, Horn (det T , /\ r WtJ) is at least one dimensional. By the upper 
semicontinuity of cohomology (see Kobayashi [Ko87]), there exists a non-trivial holomorphic 
map det J 7 — ► /\ r Wq. Let I be its image. Since det T is stable (it is a line bundle), we have 

r 

p(I) > p(detT) = rp{T) > r/i(W) = rfi(W ) = W ). 

Hence /\ r Wo is not semistable, and neither is Wo. □ 

Summing up, we proved 

Proposition 3.6. Let p : T — ► SU(p, 2) be a maximal representation and let E = V © W 
be the associated S\J(p,2)-Higgs bundle on M = F\W™. Then W is a semistable holomorphic 
bundle. 

The semistability of W is a very strong property and Theorem 3.3 follows from 

Theorem 3.7. Let T be a torsion-free uniform lattice in SU(m, 1), m > 1, and p : T — > 
SU(p, q), p > q > 1, be a reductive representation. Let E = V (B W be the associated SU(p, q)- 
Higgs bundle on M = F\W™. Assume moreover that W is semistable. Then |degW| < 
degfi , with equality if and only if m < p/q and, up to conjugacy, p is induced by the 
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rn+l 

maximal embedding f m ax '■ — > X or by its conjugate. 

Proof. Consider f3 :W (&T 1 — ► V and argue as in 3.3.1 with T ' = W and V2 = V to get the 
bound deg W < deg fi 1 , with equality if (5 is injective and W © Im (5 has degree zero and 
hence is a polystable Higgs subbundle of E. Now, we have the 

Lemma 3.8. If m > 1 and (3 : W © T 1 — > V is injective, 7 vanishes identically. 

Proof of the lemma. This is again a consequence of the relation [9, 8} = 0, which in our 
case reads (3(X)^(Y)v = P(Y)-y(X)v for all X,Y G T 1 and all v G V. Let {wi . . . , w q } 
be a basis of W above some point x G M. We can write j(X)v = Ylt=i v)wi and 
7(y)u = Ylt=l ^i(Xi v ) w i- But this implies that 

q i 
J2*i(Y,vMX) Wi = J2 x i(X,v)(3(Y) Wi . 

i=l i=l 

Since m > 1, we can take X and Y to be linearly independent and the injectivity of (5 implies 
that f3(X)wi, . . . , f3(X)wq, (3(Y)wi, . . . ,/3(Y)w q are linearly independent vectors in V, hence 
that Xi(X,v) = \i(Y,v) = for all i, namely that 7 = 0. □ 

Therefore deg W = degO 1 implies that 7 = 0, hence that c? ' 1 / = 0, i.e. the harmonic 
map / is holomorphic. The theorem easily follows. We know from the Alhfors-Schwarz-Pick 
lemma that f*gx < qg- But this implies that the inequality (f*LUx,w) < 2mq is pointwise 
true whereas deg W = ^^-degO 1 means that r(p) = ^ f M (f*wx,u)) dV = qVol(M), so that 
in fact f*gx = qg holds everywhere. Proposition 3.2 yields that / = f ma x, up to composition 
with an isometry of X . 

To get the inequality deg W > — ^q-j- degO 1 , consider the map 7 : V — > W® Q . We have 
degF = degKer7 + deglm7. Since Ker7 is ^-invariant, degKer7 < 0. By semistability of 
W, deglm7 < rk7(|degW + ^degO 1 ). Hence 

grk7 deg^ 1 q , 

deg V < : < degSZ 

g + rk7 m m + 1 

with equality if and only if rk7 = gm, i.e. 7 is generically onto, and degKer7 = 0, i.e. Ker7 
is a polystable subbundle of E. 

Again, the fact that [9, 9] = yields that (3 = 0, i.e. / is antiholomorphic: 
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Lemma 3.9. If m > 1 and 7 : V — ► W (g) «s onto, /? vanishes identically. 

Proof of the lemma. Let X G T 1 and u; G W. Let a G Or, a / 0, be such that 
a(X) = 0. Take v G V such that 7(f) = w ® a. Then for all V G T 1 , we have 
on the one hand P(X)^y(Y)v = (3(X)(a(Y)w) = a(Y)[3(X)w and on the other hand 
fi(X)j(Y)v = 0(Y)j(X)v = a(X)P(Y)w = 0. We may find Y such that a(Y) ^ 0, for 
m is greater than 1. Hence f3(X)w = 0. □ 

The rest of the proof goes like in the holomorphic case. □ 

3.5. Proof of Proposition 1.2. 

We use freely what has been done in Sections 3.2 and 3.3. If the representation p is not 
reductive, we consider its semi-simplification instead. Considering the polystable Higgs bundle 
E = V © W associated to p, we want to prove the inequality 

\degW\ < — — . 

p + 2 m 

The proof again depends on the type of system of Hodge bundle we obtain by deforming E 
via the C*-action. 

3.5.1. System of Hodge bundles of type V\ — ► W — ► V2 — > 0. 

As in Paragraph 3.3.1, we need no restriction on the rank of W here. So let q = rkW > 1. 

We use the method of Viehweg and Zuo [VZ05]. They work with a binary system of Hodge 
bundles (Vi = 0) so we explain how to adapt their proof to the ternary case. We try to fit to 
their notations as much as possible. Dualizing the Higgs bundle if necessary, we may suppose 
that degVF > 0. We also suppose that no subsheaf of V2 has a slope equal to zero. In fact, 
each subsheaf of V2 has non positive slope because 6\y 2 = and if its slope is equal to zero, 
then the Higgs bundle splits as a sum of two polystable Higgs bundles of degree zero with one 
contained in V2. 

Let us consider the Harder-Narasimhan nitrations [VZ05, HL97] 

= W° C W 1 C • • • C W l " = W 



and 



= v 2 ° C vi C • • • c vi' = v 2 



of W and V2. Let / be the maximum of all j verifying p(W^ /W^ 1 ) > 0. Remark that I > 1 
because p{W x ) > p(W) > 0. 

We construct by induction two sequences 

= jo < h < ■ ■ ■ < 3r = I and = j' < j[ < ■ ■ ■ < j' r < l' 
in the following way: 

Suppose that jk-\ and j' k _ 1 are defined. If jk-\ < I, let j' k be the minimal number with 

(3(W jk - 1+1 ) C v( k (8) n 1 , and j k be the maximum of all j < I verifying p(W') C v( k ® fi 1 . 
Then, we have non trivial morphisms 

Because of the semistability of all involved sheaves, we get 

'W^-i+ 1 \ ( vf k 
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for each k. We set E k = W jk © v( k . The sequence {E k ) < k < r defines a filtration of W jr © vf 
by Higgs subsheaves, and we denote the successive quotients by F k = E k /E k ~ 1 = F^ © F k 2 , 

where F^ = W jk /W jk ^ and F k 2 = vf/V^ 1 . 

From the properties of the Harder-Narasimhan filtrations, we have 

and 

In particular, we get for all 1 < k < r, 

^)-^F k 2 )<^) 



and 

fi(F^) > n(F&) > ■ ■ ■> fM(Ffo) > > > > • • • > f,(F^ 2 ). 

Viehweg and Zuo then define the following quantities: 

• c fc = degF fc , 

.^ = ^(F k ,),^= f i(F k 2 ), 

• rf = rk(f*), 
•r?=rk(f*)-4. 

With these definitions, we can write the above inequalities 

HY ~ < M^ 1 ) for all 1 < k < r, and $ > ■ ■ ■ > ^ > > ^ > ■ ■ ■ > $ 2 . 



We verify the properties I IV of Claim 2.2 in [VZ05] (recall that in the present situation, 
we may have j r < I"): 

I. Each r, 2 = — k v k and hence is positive. 

II. Here we need some adaptations. For each < k < r, the number Yli=i °i 1S non positive 
because E k is a Higgs subsheaf. Moreover (if we set p\ = rkVi) 



pi + E r k 2 + rk MM*) - p = IX 2 - E rk 



k=l 



k=l 



k=l 



E ^k 
u V2 
k=l Vk 



1 r r—l ,y% 



i=l 



< 0. 



Thus, p > Pl + ELi ^ 2 + rk OVtf) > ELi r\ 
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III. By assumption, degV < and therefore 



> 



ELi rk (F& + deg (V 2 /V^ ) + deg V, 

Y r i rV2 

ELi ^ + EUcfc + de § gW*^) + de § g 

ELi^>fe 2 -deg(^/lV') 

l^k=\ ' fc 
y^r V 2 V 2 
l^k=l 1 k ^k 

l~ik=l ' k 



(we use ELi c fc + deg (Vz/Vf) + deg Vi = -deg (W/W l ) > 0). 
IV. Prom III, we get 

Z^k=l'k ^k 2^k=l'k Mjfc 



W V 2 
k=l ' k 



Yfk=i r k Yj 



The r.h.s is bounded from above by max{/i^ — , 1 < k < r} (see [VZ05] for the proof of 
this), hence 

pq m 

3.5.2. System of Hodge bundles of the type V\ — >W\ — >V 2 — >W 2 — ► V3 — ► 0. 
If /3i ^ and 72 7^ 0, we have exactly as in Section 3.3.2 that 

|deg^|< 3rkF2 + 1 ^ 
1 g |S 2(rkF 2 + l) m 

using that rk72 < rkl^ instead of rk72 < m. This inequality is stronger than the one we 
want to prove here. 

If /?i = or 72 = 0, the Higgs bundle splits as the sum of two polystable Higgs bundles of 
degree 0, for example V\ — ► W\ — ► and V 2 — ► W 2 — > V3 — ► 0. Since rk Wi = rk W 2 = 1, 
we have by the previous paragraph that 

\A T ,,^ rkVj deg ft 1 . rr\i ^ rk^+rk^ degO 1 

deg Vi\ < and deg {V 2 V 3 < 

rkvi + 1 m rk 1/2 + rk 1/3 + 1 m 

which, in view of the following lemma, gives the result. 

Lemma 3.10. Let pi,p 2 , qi, q 2 be positive numbers. Letp = p\+p 2 andq = qi+q 2 . Then 

Piqi P2Q2 < pq 



pi + qi P2 + Q2 p + q 

with equality iff piq 2 = P2Qi- 
Proof. 

pq piqi p 2 q 2 (piq 2 - p 2 qif 



p + q pi + qi P2 + q2 {p + q){pi + q\){p2 + q2)' 

□ 

4. The case G = SO (p,2) 

In this section we prove the main theorem in the case where G = SOo(p, 2), the identity 
component of SO(p, 2). 
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4.1. The Hermitian symmetric space SO (p, 2)/(SO(p) x SO(2)). 

Here, the symmetric space <Y is naturally seen as an open subset of the (real) Grassmannian 
of 2-planes of a real vector space, which makes the complex structure a bit more difficult to 
understand. Again, details are to be found in [HeOl, Sat80]. 

Let Er be a real vector space of dimension p + 2 (p > 3), endowed with a non-degenerate 
quadratic form S of signature (p, 2). The symmetric space X is defined as the space of all 
2-dimensional real subspaces of Er on which S is negative-definite. It is an open submanifold 
of the real Grassmannian of 2-planes of Er. 

Let us fix a 2-plane Wr of Er on which S is negative-definite and let Vr be its orthogonal 
complement. We also fix an orientation on Wr. 

The group G = SOo(p, 2) acts transitively on X by analytic isomorphisms. The isotropy 
subgroup K of G at Wr is identified with the maximal compact subgroup SO(p) x SO (2), and 
hence X can be identified with SO (p,2)/(SO(p) x SO(2)). 

Let q be the Lie algebra of G, t C g the Lie algebra of K and q = t © p the correspond- 
ing Cartan decomposition. Let us fix an orthonormal basis (e±, . . . ,e p ) of Vr and a direct 
orthonormal basis (e p +i,e p +2) of Wr (with respect to Sry K , respectively £|w R )- We have the 
following matrix expressions: 

P = { ( ^ o ) ,Ae M ^w} — K Hom(Wi 

The tangent space T Q X at o = Wr E X will be identified with p. The tangent bundle 7Vt 
of X is the bundle G x Aciff P associated to the .ff -principal bundle G — ► X = G/K via the 
adjoint action of on p. 

Since K respects the decomposition Er = Vr© Wr, the vector bundle £r on X associated 
to G — ► X via the action of K on Er naturally splits as the sum Vr © Wr. 

Moreover, there exist two natural complex structures belonging to SO(/SW H ) = SO(2) on 
the 2-dimensional real vector space Wr, and only one that we call /, such that the orientation 
of the basis (Iw,w) of Wr coincides with the fixed one (for any non-zero w E Wr). In the 
above basis of Wr, the matrix of / is given by 

1 
-1 



The complex structure / then defines a complex structure on the vector bundle Wr that we 
still denote by /. 

Finally, using the identification TX ~ Hom(WR,VR), we get the complex structure J on 
TX : if X is a section of Hom(WR, Vr), JX = X o I. The G-invariant Kahler metric gx on 
X is defined at o by 

g x (X,Y) = tr(YX) = 2tr {'BA) , if X = f t ° ^ \ ,Y = f t ° ^ \ e p. 

The corresponding Kahler form will still be denoted by ujx = 9x{J-i •)• 

Next, we consider the complexifications V, W and E = VffiW of Vr, Wr and Er respectively. 
We extend the quadratic form S to E and the complex structure I to W by C-linearity and still 
denote them by the same letters. Let W 1 ' (resp. W ' 1 ) be the eigenspace of / corresponding 
to the eigenvalue i (resp. — i). These two eigenspaces also are the two isotropic lines in W for 
the quadratic form S\yy. Moreover, W ' 1 may be identified with (W 1 ' )* by the mean of <SW, 
and V* may be identified with V by the mean of 5W. 
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Let us define e' p+1 = ^( e p+i + ^,+2) and e' f 



P+2 ~~ V2 



: p+i 



le 



P+2 J 



In the sequel, we shall 



use the basis (e\, . . . , e p , e' p+1 , e' p+2 ) of E. The quadratic form S then writes 



S 








and 



c c 
l c 
*C 



-1 



, C £ Af Pl i(C) 



The complexifications Gc of G and -Kc of K are respectively SO(p + 2, C) and SO(p, C) x 
SO(2, C). The Lie algebra Qc of Gc splits as 6c © Pc where 6c is the Lie algebra of Kc and 



pc = P ® C 



C fl 
*D 

*c 



C, D £ M p ,i(C) > ~c Hom(W M , V R ) (8) C = Hom c (W, V) 



The two eigenspaces of the extended complex structure J (g) Id acting on pc are 



and 









c 




P = 









i) 






\ *c 





















' 











l D 











, 








C G M Pi i(C) } ~c HomcCW 1,0 , V) ~c Hom c (V, W u,i ) 



D G M P) i(C) V ~ c Hom c (W°' 1 ,V) ~ c Homc^W 1 ' ) . 



The complexified tangent bundle T C X of X is isomorphic to GxAdifPc — (GxkKc) x AdK c 
pc, whereas the holomorphic tangent bundle T 1,0 X is isomorphic to G ^AdK p 1 ' — (Gxjc 
^c) x Ad_ftT c P ■ There is a natural Hermitian metric on the holomorphic tangent bundle of 
X given on To'°X ~ M p ,i(C) by h{C,D) = 4tr ^DC). 

The holomorphic sectional curvature for the complex line (X) generated by a nonzero 
/ C C 
X = [ *C 

V *C J 2 ('COY 

is pinched between —1 and —1/2. The metric gx is Einstein and its Ricci curvature tensor is 

v 

-%9x- 



e T Q X is given by K{(X)) 



1 feci 2 



. It is clear that K((X)) 



4.2. Toledo invariant and SOo(p, 2)-Higgs bundles. 

Let p be a representation of a (torsion free) uniform lattice T of SU(m, 1) into G = SOo(p, 2), 
p > 3. We will assume that p is reductive. If it is not, just replace p by its semi-simplification 
p ss (see Paragraph 3.3.3) in the following to get the result. 

Let (Pk c ,0) be the G-principal Higgs bundle on M associated to p and / as in section 2 
and let E be the holomorphic vector bundle on M associated to Pk c via the action of Kc on 
E. The bundle E splits holomorphically as the sum of the rank p subbundle V = Pr c Xk c ^ 
with the rank 2 subbundle W = Pk c *k c W. But, in the present situation, we get more 
structure on W, because Kc also respects the decomposition W = W '° © W ' 1 . This implies 
that if we call L the line bundle Pk c x^ c W 1,0 , W holomorphically splits as the sum L©L _1 . 
Moreover, since Kc preserves S\y, we can identify V* with V. In particular, deg V = 0. 












( ° 





i 


, where / 
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The Higgs field 9 is a holomorphic (l,0)-form taking values in the bundle Pk c XAdK" c Pc — 
Hom(L, V) © Hom(L _1 , V) so that we can write (see also [BGG05]) 

(3 : L — ► 

-f-.L- 1 — ► v^n 1 

The Higgs vector bundle (E, 9) is polystable. 

The bundle f*T 1,0 X is isomorphic to the bundle Hom(L, V) and therefore its degree is 
given by — pdegL. We obtain that r(p) = ^jdegL. Hence the main theorem in this case 
follows from: 

Theorem 4.1. |degL| < ^degQ 1 , that is \r(p)\ < Vol(M). In particular, when m > 1, 
a representation p : T — ► SOo(p, 2) is never maximal. 

Proof. We shall denote by 9 2 the morphism of vector bundles 

9 2 : T 1 xT 1 — ► End(£) 

(X,Y) .— 9{X)o9{Y) ' 

by *7/3 the morphism 

*7/3 : T 1 x T 1 — ► End(L) 

( X ,Y) t 1 {X)o(5{Y) ' 

etc. 

We remark that the system of Hodge bundles obtained after deformation of the Higgs bundle 
(E, 9) via the C*-action on the moduli space is very simple. Indeed, the limiting (E, 9) must 
verify 9 n = for some n. In particular, as 

/J*7 + 7*/9 
*7/3 *77 
*/3/3 */3 7 



the (2, 2) matrix 



W(X,Y) ^ 7 (X,Y) J \W(X,Y) *Py(Y,X) 



(we use 9 2 (X,Y) = 9 2 (Y,X) and in particular t 0j(X,Y) = t (5^{Y,X)) must be trace free for 
any X,Y G T 1 , which implies *7/3 = (and */?7 = 0). Thus 

/?*7 + 7*/? 
'77 

*/?/? 

Suppose now that there exist X,y G T 1 such that Vy(X,y) / 0. For any X',Y' G T 1 , 
*/3j9(X / ,y / )*77(X,y) = because 0" = 0, so we conclude that for every cc G M, either 
*/3/3 = or '77 = on T% x and then, by holomorphicity, either t f3[3 = or *77 = on 
T 1 x T 1 . 

We work on the system of Hodge bundles that we just described. Suppose for example that 
t f3f3 = 0. Then, the sequence 

L-i Im 7 ^ 7 L -^-> Im (3 — > 

defines a Higgs subsheaf of (E,6). 

The bundle L © T 1 is semistable and Im /3 is also a Higgs subsheaf of E, so we have 
p(L © T 1 ) < p(lm/3) < and then degL < (if p = Q } then degL < 0). 
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Let us consider the maps 7 : L~ x (g> T 1 — ► I1117 and *7|i m7 : Im7 — > L tg) fi 1 . We call r 
and r' their respective rank. By stability, we have 



deg I1117 > r ( — deg L H deg T 1 

\ m 



and 

r' ( degL H deg ft 1 ) > deglm t 7|im 7 = deglm7 - degKer t 7|im 7 - 



in 



Using the fact that degKer*7| Im7 < 0, we immediately get degL > — dc ^ . □ 
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